Audio video data are stored on data carriers, i.e. film or tape, or transmission channels, i.e. radio or telephone, in a fixed format, which does not allow an extension by novel audio formats or other synchronous or image synchronous, respectively, supplementary services, such as subtitles. Thus, with the introduction of, for example, new audio formats thus, new data carriers or film copies, respectively, have to be produced, which have the new audio formats.
FIG. 8 shows an exemplary film 110. Film information, such as video information or images 112, respectively, which are also referred to as “frames” or “video frames”, and audio information or a plurality of analog or digital soundtracks 114, which have “audio frames” in the digital case is applied in spatial sequence or, during replay, in time sequence, respectively. Further, the film 110 has, for example, advance perforations 116, with the help of which the film is played.
Basically, two methods are known for synchronizing supplements.
The first method comprises storing a time code on the data carrier, such as with DTS (digital theatre system) for cinema sound or in an additional channel connected to the audio signal. Examples herefore are ancillary data by DAB and mp3. The time code is used to replay sound or additional information, respectively, synchronously from an external data carrier, for example CD with DTS. However, it is a disadvantage of this method that every additional format needs further space on the data carrier or transmission channel, respectively, which might not be available. With film, these are for example the tracks for analog sound, Dolby digital, DTS, SDDS (sony dynamic digital sound). However, proprietary formats avoid the utilization of the time code of one extension by other extensions. Mutual interferences of the extensions cannot be avoided, one example is the usage of ancillary data in MP3 for additional information and bandwidth extension from different manufacturers.
The second method is based on the improper use of analog soundtracks for storing time code, such as it is used for example in a prototype cinema equipped with an IOSONO system. However, it is a disadvantage of this method that the analog track exists in all systems and is often used as fallback solution during interferences of the other systems, which means a misuse of the analog track prevents the fallback possibility. Automatic switching to the analog track, which is installed in most cinemas, leads to the fact that the time code is replayed as analog signal when no signal is present on the “modern” tracks for Dolby digital or DTS, respectively. Thus, in the prototype cinema, during a pure wave-field synthesis reproduction, which will be discussed below, the redundant analog reproduction has to be switched off manually, because otherwise the time code can be heard via the redundant further loudspeakers.
The acoustic wave-field synthesis, short WFS goes beyond the surround approaches of the formats Dolby, SDDS or DTS. In WFS, an attempt is made to reproduce the air vibrations of a real situation, which constitute sound, across a whole room. In contrary to conventional reproduction across two or more loudspeakers, where the mapping of the position of the original sound source is limited to a line between the loudspeakers, the wave-field synthesis is to transmit the whole sound field true to the original to the room. This means that the virtual sound sources can be exactly spatially localized, and even seem to exist within the room, and thus can be encircled. Systems with up to 200 loudspeakers in cinema systems and up to 900 loudspeakers in theater sound systems have already been realized.
Wave-field synthesis is based on the Huygens' principle, which says that every point on a wave front can be seen as starting point for an elementary spherical wave. By interference of all elementary waves, a new wave front occurs, which is identical to the original wave.
Such a sound system has been developed by Fraunhofer Institute for Digital Media Technology under the name IOSONO and is used in cinema of Ilmenau.
Thus, the cinema of Ilmenau is mentioned as a practical example, where the wave-field synthesis is operated in two modes.
In the first mode, the cinema is operated as “real” wave-field synthesis system, wherein the time code is stored on the analog track of the 35 mm film, such as has been discussed above with regard to the second “improper” method, where the WFS sound is played from an external medium, such as hard disk or DVD.
In the second mode, “compatible reproduction”, the sound stored in every 35 mm film is read out and decoded by a Dolby processor, alternatively, DTS or SDDS, respectively, could be used, wherein the Dolby processor, if necessary, switches automatically to the analog track and maps the occurring multi-channel signal via WFS to virtual loudspeakers.
Since different signal paths are needed for both modes, a division of the signal coming from the read head for the analog signal is necessary, which causes additional technological effort.
Thus, in summary, it can be said that there is no room on current spools of cinema film to attach a further synchronization track, such as for external sound systems or subtitle systems. All cinema sound systems available up to now, analog and digital, obtain their soundtrack either directly via one or a plurality of soundtracks on the spool of film or by a manufacturer-specific time code signal on the spool of film. This means that for both known approaches, as explained above, new copies of the films have to be produced, usually with significant costs. Yet, audio formats like Dolby digital and SDDS allow modern audio experiences, but have still no time code for the synchronization of, for example, subtitles or foreign-language versions of the film sound recording.
Hence, Frank Jordan and Jesper Dannow, in their publication “Generating Timecode Information from Analog Sources”, 118th Convention, Audio Engineering Society, of May 28 to 31, 2005, in Barcelona, Spain, Convention Paper 6473, propose generating a time code on the basis of the analog sound track. The publication describes a system with the designation “Soundtitles”, which is attached to the analog sound track of the projector. Based on an edited, digital copy of the sound track and the analog signal of the film projector, time information or a time code is determined by cross-correlation. The system “Soundtitles” consists of three components. The core module “Sync Tracker” generates the time code signal. The second module, the “Sync Player”, generates subtitles projected with a beamer, for example. The third module, the “Clip Player” plays synchronized audio clips transmitted to the cinema visitor via wireless headphones.
It is disadvantageous, however, in the “Sound Titles” system that no lip-sync can be produced, because there still remains a time offset of almost 100 ms. A time offset of about 100 ms between image and sound is, however, perceived as disturbing by the viewer.